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EVALUATION OF CLASSROOM TEACHING INNOVATIONS AT THE 

UNIVERSITY OF SEVILLE 

L. M. Villar (Dir.), J. Aceval, F. Caro, C. Casanueva, J. Correa, S. Gutierrez, C. Hervas, 

D. Maldonado, E. de Manuel, J. Munoz, E. Padilla, R. Perianez, C. Romero and A. 

Vazquez 

University of Seville (Spain) 

ABSTRACT 

This study describes and compares twelve classroom-teaching innovations that 
were introduced at the University of Seville during academic year 1999-2000. Students 
completed the Student Demographic Questionnaire (S.D.Q.) and we assessed their 
perceptions of the classroom learning environment with the Evaluation of University 
Teaching Activities Questionnaire (E.U.T.A.Q.). In addition, University classroom 
innovation practice was analysed through teaching plans, observations and semi- 
structured interviews. Significant differences were found in several factors of student 
biography. Findings shed some light on a neglected theme: the teaching practice of 
University faculty. Research results and implications for improving University teaching 
innovation are discussed. 

1. INTRODUCTION 

This study assesses twelve classroom-teaching innovations in different subject 
areas, introduced at the University of Seville during academic year 1999-2000. 

Universities are deeply concerned about developing the quality of classroom teaching 
and student learning abilities and they take great interest in improving classroom 
teaching. Current Spanish University teaching research reflects three educational 
assumptions: (1) current practices in classroom teaching evaluation; (2) the use of 
teaching questionnaire systems for staff development, and (3) the need to make 
improvements in classroom teaching evaluation practices. 

The most common method of evaluating teaching in Spanish universities is by 
means of student surveys. Any attempt at the evaluation of teaching must address the 
following research question for students and professors within the higher education 
system: ’’What is teaching?”. Student perceptual measurement of teaching represents the 
major approach of this article. First, perceptual assessment is based on different 
University students’ experiences over a one-semester or a one-year course. Second, 
perceptual assessment entails giving a succinct account of the satisfaction judgments of 
all University students in a class group. Third, perceptual assessment is associated with 
students’ demographic characteristics and background factors (e.g. course choice 
concerned with the gender, age, geographic location, class size, prior course grades, 
work interests and other background and experiences of the students involved). Fourth, 
perceptual assessment outlines students’ interpersonal relationships as a prelude to 
enhancing their academic focus and, hence, satisfaction with the class social life. Fifth, 
perceptual assessment may be used for feedback on professors’ innovative teaching and 
learning processes in the form of profiles when coupled with other improvement 
strategies (e.g. staff development programmes focusing on improving skills, interactive 
approaches and establishing an environment - physical and social - to support the 
achievement of high quality student learning). Sixth, perceptual assessment of 
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psychosocial characteristics of classrooms is a relatively valid source of criterion 
variables of curriculum and teaching quality (e.g. creating a defensible climate 
standards-based assessment). Seventh, perceptual assessment of University classroom 
learning environment is targeted at the communication process, instead of being a kind 
of personnel evaluation of the professor who teaches a course. And, eighth, perceptual 
assessment of classroom climate is well supported by empirical research (Fraser, 1999). 
In addition, professors are teaching researchers who construct and interpret class actions 
and their own voices and beliefs. 

/. 1. Research purposes and hypotheses 

The purpose of the present study is twofold. Our first objective is to measure 
University students’ perceptions of classroom climate and to compare their classroom 
climate dimension scores with those of University students with different background 
factors. Any scheme for the evaluation of a teaching innovation is likely to have suitable 
implications for classroom practices and social values. Students in the study are enrolled 
in a discipline course that emphasized methodology and pedagogy and systematically 
incorporated innovative aspects of teaching in the curriculum. We hypothesised that 
student’ classroom climate dimension scores measured by the Evaluation of University 
Teaching Activities Questionnaire (E.U.T.A.Q.) (see Appendix I) would be significantly 
different to those of University students with other demographic characteristics and 
background factors (e.g. course level. University department, type of subject, gender, 
age, etc.). (See Student Demographic Questionnaire (S.D.Q.) in Table I). Students as 
recipients of innovation are academic and socially diverse, and may determine the 
process of a teaching innovation. At least 14 hypotheses on the studying and learning 
approaches of college students were tested. 

(Table I should be put about here) 

A second objective of this study is to develop a University teaching theory that 
is grounded in data systematically collected and analysed (e.g. professors as teaching 
researchers are constantly comparing pieces of classroom teaching information) (Strauss 
& Corbin, 1994). Professors participated in the study on a voluntary basis. We 
hypothesised that professors would make a University teaching model induced from 
diverse practice data sources. 

1.2. Related Literature 

The general literature on students’ assessments of classroom climate has been 
growing for more than a decade (Fraser & Walberg, 1991; Aldridge & Fraser, 2000). 
Researchers and practitioners are demonstrating a significant growing interest in this 
approach to evaluating University classroom climate in Spain (Toledo, 2000). However, 
no extensive empirical support for the accuracy of Spanish University students’ 
judgments based on climate assessment is available. Evidence (derived largely from on- 
demand University teaching quality assessment) is accruing on the potential of 
classroom learning environment assessments to improve University teaching and 
learning. Use of classroom climate questionnaires in a Pedagogy discipline class 
resulted in reflective changes in learning and instruction (Villar, 1999). Other 
University practitioners also reported that classroom climate assessment led to better 
curricular integration (Villar, 2001a). In addition, multilevel analysis strategy has been 




3 



used in a Spanish study to investigate College size effects and student learning 
environment perceptions (Villar, 2001b). 

2. METHODOLOGY 

2.1. Description of the Students 

Table I presents the population of this study and the list of variables contained in 
the Student Demographic Questionnaire (S.D.Q.). It comprises 665 University students 
belonging to twelve innovative subjects within eleven departments. We recorded 
student background, School and University characteristics, working conditions and 
home residence. Also, we transformed into continuous all independent variables (e.g. 
Course level. Age, Complementary jobs, etc.) for ease interpretation. The group of first 
course level students represented 61.7 percent of the population. Students belonging to 
the School of Geography and History were the most numerous amounting to 39.2 
percent. Some sixty-eight percent were women. The 18 to 21 year-old group of students 
represented 68.4 percent. Students who had been enrolled on a Pre-University Sciences 
Course made up 32.9 percent. A large majority of students, 74.4 percent, reported that 
they had studied in a state High School, and the vast majority, 73.2 percent, said that 
they lived in Seville. Students performed well in College grades. More than 44 percent 
said that their academic background was notable (e.g. a grade above pass qualification). 
As for future aspirations, 43.3 percent of students preferred to work outside of the 
public administration. Most students did not have a scholarship (57.1 percent). The vast 
majority, 93.1 percent, reported that they were not repeating the subject (e.g. they had 
not failed the subject in a previous academic year course). Likewise, most students had 
not failed other subjects (74.6 percent). 

2. 2. Instruments Used for Data Collection 

The core quantitative instruments in this study were (a) the Evaluation of 
University Teaching Activities Questionnaire (E.U.T.A.Q.), which encompasses 25 
items in accordance with principles of cognitive and social psychology, to include ten 
learning dimensions from the cognitive apprenticeship literature. It addresses students’ 
co-construction of knowledge, and professors’ scaffolded orientation. In another study, 
statistics were used to determine the reliability and validity of E.U.T.A.Q., including 
internal consistency reliability by calculating Cronbach’s alpha coefficient for all iems 
{a = 0.8635 ); examining item intercorrelations and revising scales for final use, and the 
seven-factor solution produced the most reasonable description of the item structure 
(59% of the total variance was explained) (Villar, 2001b). And, (b) the Student 
Demographic Questionnaire (S.D.Q.), which is composed of 17 items. This instrument 
taps selected students’ biographical factors (i.e., standard demographic and academic 
characteristics - age, sex, course level. University department, type of subject, etc. -). 
These factors were chosen as independent variables to meet the first objective of this 
study. 



After site selection, a number of observers (39), made up of undergraduate 
students in the field of Pedagogy, wrote down 79 class observations as narrative 
vignettes, and interviewed 84 students. Then, we selected qualitative methodology to 
obtain the rich descriptive, narrative, and personal stories that emerge when an 
innovation is explored in its natural setting. All undergraduate students were trained 
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specifically for this study. They also collected other artefacts of the practice, including 
professors* lesson plans, and innovation diagrams and sketches to better understand the 
meanings with which professors imbue their teaching experiences. In addition, they 
administered the E.U.T.A.Q. to the students. The professors helped interviewers to 
select about seven students in each innovation as target students. Semi- structured 
interviews on how students were learning the content areas in the light of the innovation 
were the main focus. In the dialogue, ethical considerations (e.g. consent, privacy, etc.) 
with the students were maintained. Nevertheless, each interviewer made an interview 
script for each student (see Appendix 2) that was adapted to meet the situation. Also, 
University professors answered questions about the purpose of their innovation from an 
interview script (see Appendix 3) made up by the principal researcher. All observations 
and interviews were transcribed by the undergraduate Pedagogy students and coded by 
the twelve professors. 

2.3. Innovations 

The success of a University teaching innovation is dependent to a large extent on 
professors feeling close to the origin of the project. It follows then, that all twelve 
professors were concerned with teaching innovation evaluation from its beginning, thus 
supporting a democratic and participative approach. Professors participating in the 
innovation evaluation group were aware of the metaanalytic process of on-going and 
monitoring exercises, which gather and analyse all information. The twelve professors 
and the principal researcher had group meetings, semi-structured interviews and 
informal conversations to focus on evaluation processes and issues (e.g. coding 
processes, personal views). 

2.4. Analysis of Data 

SPSS for Windows was used for the Analysis of variance (ANOVA) tests that 
allowed us to discover the significance of the 19 student demographic hypotheses 
related to the first objective of this study. Qualitative analysis of the content (second 
objective) was condensed by means of codes, made up of 16 built-up declarations that 
were defined by the twelve professors as co-authors of the report, that were developed 
to accomplish the empirical categorizations of the texts (see the Category System in 
Appendix 4). Codes highlighted teaching practices that were connected to the 
innovations. Hence, a variety of data sources and a methodological triangulation of class 
observations, professors and students’ interviews and student perceptions were used in 
this study. 

3. RESULTS 

3.1. Descriptive Analysis 

The initial findings concerning students’ agreements for each learning dimension 
are presented in Table II, which lists percentages of agreement-value of the E.U.T.A.Q. 
scale, means and standard deviations for all twelve University innovation class groups. 
(See means profile for E.U.T.A.Q. dimensions of class innovations in Figure 1). First, 
noteworthy here is the priority that respondents gave to such altruistic environment 
subscales as Connections and Clarification (mentioned by a third of the population). In 
contrast. Student autonomy was not particularly high. And second, there is some 
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indication in the subscale means that most students were involved in a known 
environment, which tended to portray a kind of vague and imprecise climate. 

(Table II should be put about here) 

(Figure 1 should be put about here) 



3.2. Inferential Analysis 

One-way ANOVA on each E.U.T.A.Q. dimension revealed a reliable difference 
between groups, according to Course level and University School. F-statistic yielded 
significant results in a very reduced number of dimensions (only 5) included in Type of 
Pre-University Course, Academic background, and Complementary activities while 
studying. The groups with different Age, Complementary jobs while studying (you help 
doing tasks at home), and, those that Repeat subjects in other courses were significantly 
different in four class climate dimensions. To examine the significant effects more 
closely, students who repeat this subject were distinct in two environment dimension 
perceptions. 

Additionally, a significant F test was obtained in only one dimension among 
those students who were different in Gender, and who had Complementary jobs while 
studying (you teach children). Complementary jobs while studying (you work in an 
office). Future expectations and selected the field studies at the University in a given 
priority. The findings on student motivation and teaching interrogation / discussion 
variation underscore the importance of early professor competency acquisition when 
innovating teaching (see Table III). 

(Table III should be put about here) 

These findings provide some support for the hypothesis that University student 
will benefit cognitively from a rich learning environment when professors offer 
innovations including many developmentally appropriate activities for classes 
composed of heterogeneous student factors. No other effects were significant in the 
analysis which examined differences in the perceptions of student groups according to 
such variables as Type of High School Centre where you carried out Secondary 
Education, Residence during the week. You receive some type of scholarship in your 
studies, and Housing during the course (see Table III). 

2.2. Analysis of Qualitative Data 

We have proceeded as in the collages or pictorial assemblages mixing professors 
and students’ actions and declarations making up a painting or map of the general 
teaching innovation where the fragments that emerge from the College reality are 
illustrated by codes giving essence to the innovation sample. Professors have elaborated 
a theoretical model (Figure 2) derived primarily from examination of transcribed 
materials (class observation vignettes, clinical interviews, and so far) that maps key 
codes as specific features of a University teaching committed to excellence as well as 
their antecedent notions and consequence effects. The model suggests that professor 
excellence is an extension to the contextual level. Professors have differentiated discrete 
pedagogical knowledge and have interpreted instructional strategies that are constituted 
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by class processes and contextual factors through which they assess teaching 
innovations and measure learning results. To illustrate the categories of the model, we 
show particular professors’ understandings of the nature of innovation. On both 
theoretical and empirical grounds, we see the teaching framework as useful for the 
assessment of underlying innovation teaching structure and changes in professors’ 
cognitive structure. 



(Figure 2 should be put about here) 

3.4. Qualitative findings: conceptual framework of class innovation teaching 

The Instructional Strategies show an inveterate yearning for Inquiry (IND), as is 
demonstrated in the following paragraph of a class of Primary Teacher Education, (as 
has been proven in hypotheses 1, 2, 3, 5, 6, 9 and 18, class discussion was perceived 
different among students for distinct student biographical factors): The professor began 
to alter the activity to see what modifications the figure suffers when the order is 
changed. The professor invites a student to solve the new position. He explains logically 
how the changes take place according to where the turtle looks. The professor 
recaptures what has been done previously to recover the acquired knowledge and to find 
the logic to the problem, and he says, "So that you understand it". And the professor 
continues saying, "Do you understand it? Is it complicated? Well. Make a conjecture 
and try to see if it is right". About 7:20 p.m. the professor says that they may work with 
the program for the last ten minutes of the teaching period. Immediately he proposes a 
new activity so that students do it alone, but the professor continues directing it. (New 
Technologies in the Teaching of Mathematics. Observation vignette 1). 

The intellectual and emotional relations in working groups (TGR) characterized 
a kind of class teaching style. A student of Architecture Constructions II answered: Yes, 
we work in groups; we also use groups in other subjects like Construction and History; 
right now I don't remember others; for other tasks we don't usually form groups, nor do 
we get involved in this style of class dynamics. To me, groupwork contributes to seeing 
other people’ approaches, to see if I’m mistaken, or right, to share knowledge, not only 
to see how I face up to the tasks, to feel support... Not everything is to receive, but 
rather it requires a little bit of responsibility, mainly responsibility. There are people 
who believe that working in groups consists of some people doing the whole task and at 
the end everyone signs the paper. (Task attribution based upon students' learning styles. 
Individualization. Student interview 1). 

The presence of these categories makes patent the full expression of a teaching 
that breathes in the depth of inquiry and reflects on the students’ working group 
understanding. To this aim Teaching skills and techniques affirm their presence before, 
during and after class teaching communication. The departure is exactly from 
Objectives / Aims / Expectations (OME) that are shovm condensed in the expression of 
the question made by a student observer to another student of a class of Business 
Administration and Marketing (as has been verified in hypotheses 1, 2, 3, 5, 6, 9, 10 and 
18, the structuring was not perceived similar by students according to their biographical 
factors): Question: Would you believe that this subject covers all the expectations that 
you had of it? Is it what you expected? Answer: The truth is that it is O.K. Professors 
said what the subject would be like at the beginning of the course. They said it would be 
very practical. Also, that they would introduce us into a company; that the task would 
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occupy a lot of time..., and it’s as right as it has really been. {Development of 
managerial training tools: the case study method. Student interview 1). 

The appearance of the class teaching is a road made up of six categories 
wrapped by the Presentation of ideas and concepts (EXP) by means of Resources (REC) 
in a History of Art class (as has been confirmed in hypotheses 1, 2, 3, 6, 9, 13, 14 and 
18, the degree to which they understood a problem or the materials was different among 
students according to their biographical factors, as well as was the relationship of 
students’ new to prior knowledge in hypotheses 1, 2, 3, 10, 11 and 13): All slides were 
quite clear, which helped the professor’s explanation a lot, and they enabled an 
understanding, knowledge, and follow-up of the churches that were shown. {Initiation 
research activity and University teaching. Observation vignette 5). 

For that reason the class crumbles in Activities (ACT) that try to give 
participation to the student, delaying the monologues of a professor (as has been 
verified in hypotheses 1, 2, 3, 6, 9 and 10, the degree to which they made their own 
connections was deduced in different ways by students’ biographical factors that 
differentiated them); a student expressed herself in the following way in a class of 
Business Administration and Marketing: Question: Can you explain in a general way 
how you have done tasks in this subject? Answer: Summarizing the topic was the way 
we did the tasks. Later, a topic was distributed, transparencies made and we studied 
everything; it was also compulsory to present topics; in my case I had to make a 
presentation in the first session, therefore I have almost forgotten them. {Participation 
approach to business administration by means of projections and case studies. Student 
interview 2). 

Values (VAL) - born out of Group Work (TGR) - were aspiring to a higher 
reality, to a concert of notes and an opening feeling that want to be present in teaching: 
Question: Do you believe that group learning is deeper that individual learning? Why? 
Answer: If a student works in a group she memorizes much more because the group 
really resolves many issues for you and it opens other doors, where on your own you 
close down your mind and you see that you don't have any other exit; you look for 
support in your peers and you may find an open door. Problems are better solved by 
working in groups. {"Development of managerial formation tools: the case study 
method. Student Interview 1). 

Class teaching is symbolic. It is related to new technologies. Immersion is made 
in diverse territories of reality from Audiovisual Media (MAV) to Resources (REC). A 
passage on the use of MAV appears in the following declaration: The first student 
comments on the legend around the castle. When finishing the legend, while the 
videotape goes on, she explains some aspects that were shown on videotape. The 
professor warns that students should comment on the music that was accompanying the 
videotape images, and they say that it was Carmina Burana, songs from that period. 
When finishing the videotape, the second student begins to speak, while a third student 
places a transparency on the overhead projector. Only the slide that shows the fortified 
walls is projected, while the second student points out the places of interest on the slide. 
{Initiation research activity and university teaching. Observation vignette 4). 

The use of MAV has several forms of expression, as the use of REC in a class of 
Psychology that synthesizes a daily explanation (as has been verified in hypotheses 1 , 2 
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and 3, the degree to which audiovisual media was used varied according to students’ 
biographical factors). This can be seen in the following paragraph: She usually explains 
the topic with the support of different teaching materials such as the overhead projector 
or the videotape. {Role-playing of conflicting situations among handicapped students, 
their parents and the school. Student Interview 6). 

Finally, Evaluation (EVA) reels off understanding of a possible learning, that a 
professor commented to an observer in the initial contact of his class innovation: 

The evaluation will be carried out in two ways. First, we will keep in mind the quality 
of task realization; then, exposure and participation. It’s compulsory to have an 
attendance of about 80% to reach a pass qualification. {Participation approach to 
business administration by means of projections and case studies. Initial interview with 
a professor). 

The articulation of an innovation depends on the flexibility of a College and 
other class factors that converge in the Context factors. The Physical Environment (EFI) 
reflects the architecture on which the faithful image of a teaching style is: The class is 
located in one of the corridors of the School of Philology. This is longer than it is wide 
and it has capacity for 75 students, since it’s furnished with 15 benches, with five seats 
in each one of them. There is only one entrance. It’s quite luminous and in it we find 
three windows that look onto an interior patio. On the platform, where the professor’s 
table is we can observe a blackboard and to the sides a piece of furniture where the 
videotape apparatus and a screen for slides are located. Another teaching resource is a 
computer projection unit that hangs from the ceiling. {Initiation research activity and 
university teaching. Observation vignette 2). 

That is different when EFI includes new technologies: Students are sitting at the 
computers; one per computer. They switch on the computers and are being introduced 
to the program "Win-logos", with which they will work during the class period. The 
professor has not mentioned to them how they may access it. Students already know the 
program and access it without any problems. {New Technologies in the Teaching of 
Mathematics. Observation vignette 3). 

Also, class teaching-learning time has to be replayed in a curriculum innovation, 
because if we move the needles of the clock, we modify people’s effort and destination: 
Time is always the negative aspect; class periods are very long and they become very 
dull. When it’s compulsory, that is the worst aspect, since the fact of having to attend 
the classes obligatorily forces me to have to miss the rest of the classes; also, I work the 
whole morning and it messes up my timetable for the whole afternoon so I can’t 
combine other subjects. {Participation approach to enterprise administration by means 
of projections and case studies. Student interview 3). 

A curriculum innovation must engage students in activities that produce a 
positive class Climate that value knowledge as socially constructed and ethic of caring. 
In this case, the atmosphere that a professor describes when he was interviewed refers 
more to the departmental environment as a workspace: Anyway, what’s fundamental for 
the introduction of the innovation is an atmosphere of open and participative work 
where tasks well-done are rewarded, even if in an informal way, like any initiative that 
supposes an improvement in our teaching work. In our department these intrinsic 
rewards for innovative work have always been regularly encouraged. But in recent 
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times people are looking for more concrete results as much for the department as for the 
professors’ work and career, and people are leaving aside the most idealistic postures on 
teaching, to concentrate on what you can achieve and the benefits that that generates. 
{Participation approach to business administration by means of projections and case 
studies. Professor interview). 

It is not a fiction that social Relationships (REL) in the class presented as facts 
refer to the tangible atmosphere of a class. Cordiality in social relationships has the 
hope of giving students security: The professor asked the student if something was 
wrong with her, but she was silent. The fear to present a task before the entire group has 
made her so nervous that she cannot speak, so they have to take a break for several 
minutes in her lecture (...). Meanwhile, the professor tries to encourage her, calming her 
down and giving her some advice. Finally, they decide to carry out the presentation in a 
different way, interviewing the student who would simulate being an employee of the 
company. In this interview, all other group members participate. Despite this, she 
continued to find it difficult and her place was later taken by one of her peers, as she 
was left speechless again. {Development of managerial formation tools: the case study 
method. Observation vignette 9). 

The utopia of our University time is almost always limited to looking for a form 
of Collaboration / cooperation (CCO) in teaching which is a class participation pattern 
that provides the best chemistry for interaction and escapes from the competitiveness (as 
has been known through hypotheses 1, 2, 3, 5, 6, 12 and 13, the degree in which they 
reached agreement was very different according to student biographical factors). It is 
necessary tutors facilitate and develop personal and social needs, if students demand it: 
Question: What are in a reasoned way the positive and negative aspects that you see of 
this subject? Answer: Well, the most positive thing that I think is that we have related to 
the different students that we were in the group, because most didn't know each other 
and, at the end it’s not that we have become very good friends, but you simply know 
people and we have interrelated very well. And, the presentation of the task also shows 
you how to speak in public, and that helps you. And the debate has seemed very good to 
me; it’s a thing that memorizes other students’ perspectives, others’ opinions; then, you 
share ideas. You learn with the knowledge that you already have and you share ideas, 
you know? And that has seemed fantastic. {Participation approach to business 
administration by means of projections and case studies. Student Interview 6). 

Motivation or interest (MIN) is the arrow that arrives to the irreversible 
destination of personal change. University students must learn how to manage their 
learning, and instructional programs should adapt to learner differences in their 
motivation and/or interests, including student choice as a key element (as has been 
known by hypotheses 1, 2, 3, 4, 5, 13, 17 and 18, the students noticed a different degree 
of involvement due to their biographical factors). In a class situation, tutor and students’ 
adherence to teaching development is shown: Students show great interest for the topic, 
constantly participating by means of brief interventions that show their competence 
through making announcements. Later a discussion of the global task is influenced by 
suggestions, doubts, methodology, etc. In the discussion, all students participate in an 
organized way and regulate their own interventions. {Development of managerial 
training tools: the case study method. Observation vignette 2). 
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The Projection or implications (PIN) of an innovation are decorated with 
rational foundations about change in the course of an action and in the persistence of 
singularity (as has been inferred from hypotheses 1, 2, 3, 10 and 15, students 
according to their biographical factors - varied in the degree of interest). Question. 
Because students do practically everything in this subject, do they have a very active 
role? Do you believe that you learn more in this way? Answer: Yes, we have centred 
our project on the system of quality; then last year, we had a course on quality, but 
really quality in theory is very different to understand that in practice; in practice it’s 
even simpler. You see a system of quality imposed upon a company and they explain it 
to you, and you say, "I don't understand it", but really in practice, you say yes. Then we 
have informed ourselves through class notes that we had from a second-year course, and 
other books; then, it’s constant self-learning, day by day. {Development of managerial 
training tools: the case study method. Student interview 1). 

With the subtle brushstrokes of student perceptions we have painted a class 
portrait. Student Autonomy is the learning dimension that was closer to the value I don't 
know of E.U.T.A.Q. Architecture students were not aware of whether they had acquired 
responsibilities or changed their attitudes; Question: How do you value professors’ 
involvement? Answer: I believe that professors maybe go a little blindly; they try to do 
everything they can think of, regardless of whether it’s correct or not; if it’s correct they 
repeat it and if not, then they don’t; it’s all trial and error, but at least they’re trying 
something, within their limitations. On the part of the students, there are people that are 
interested and they participate, and at the same time there are people that do not care; I 
don't know if it’s because they don’t like the field of study and they’re here because 
they’ve not been able to study something else. {The organization of a thematic class of 
Architecture as a strategy of education innovation. Student interview 2). 

Evaluation was in the middle of the values Totally of agreement and Agreement 
of E.U.T.A.Q. Students coincided in appreciating that some initiatives like teaching 
innovation would significantly improve the quality of teaching. The more creative, more 
self-reflective, that is how we see professors who do not doubt to constantly undergo 
student as it happened in the Department of Business Administrations and Marketing; 
The professor begins saying, “Thank you for attending the evaluation session to those 
that have come voluntarily"; afterwards, he explains what the evaluation session will 
consist of “We want you to evaluate us; this somehow will help us for next year’s 
course; we are open to modifications that may be made to our teaching program . 
{Participation approach to business administration by means of projections and case 
studies. Observation vignette 6). 

3.5. Discussion 

We vigorously confirmed hypothesis 1 Course level (all dimensions); as 
happened with hypothesis 2 University School (all dimensions). We very weakly 
confirmed hypothesis 3 Gender (a dimension) and hypothesis 4 Age (four of ten 
dimensions); half of hypothesis 5 Type of Pre-University Course (five of ten 
dimensions); as in hypothesis 6 Academic background (five of ten dimensions); and 
also in hypothesis 7 Complementary activities while studying (five of ten dimensions); 
but, hypothesis 8 Complementary jobs while studying: You teach children was very 
weak (one of ten dimensions); and as hypothesis 9 Complementary jobs while studying: 
You work in an office (one of ten dimensions); nevertheless, hypothesis 10 
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Complementary jobs while studying: You help doing tasks at home was more powerful 
(four of ten dimensions); it was very weakly confirmed hypothesis 11 Future 
expectations (a dimension); as hypothesis 12 The field studies at this University were 
chosen... ip. dimension); it was very dim or weak hypothesis 13 You repeat this subject 
(two of ten dimensions). Finally, it was partially verified hypothesis 14 You repeat 
subjects of other courses (four of ten dimensions). Overall, the findings indicate that 
class innovation teaching structure can produce significant changes in the functioning of 
students’ perceptions. The classroom learning environment knowledge and experiences 
gained in the innovation-teaching arena are different according to University context 
factors. 



Professors never work in a vacuum. It is through constant exposure to and 
filtering through teaching innovations that professors maintain the professional 
knowledge, skill, and techniques they use to help their students learn. Much call for 
University reform has focused on changing scientific research programs, the evaluation 
of the field of studies’ curriculum, and University teaching, but this research suggests a 
need to change the “teaching setting” from a sociocognitive perspective. A University 
teaching innovation is a kind of new educational culture within which students are 
bathed in new learning rhythms. 

Moreover, all the innovations we studied exhibited all five characteristics: (1) 
fostering the inquiry style, (2) valuing dynamic and participant methodology, (3) 
creating significant tasks that provide students with interest, (4) engendering expressive 
teaching aided with new technologies to make teaching records (notes by means of 
transparencies, debates about movies, urban field experiences, cross-disciplinary 
approaches, topics and tasks close to life, etc.), and (5) coordinating efforts to improve 
University quality teaching. Overall, the professors are in touch with their students, their 
profession, their colleagues, and the University at large. 

3.6. Implications 

1. Appraisers / participants. Innovative professors testified to the lively, current 
reflection on teaching at University. As students according to background factors 
unequally perceived the effects of teaching innovations, these characteristics should be 
kept in mind when planning and offering new innovations to students. At the least, class 
interventions should be made to modify motivation and interrogation / discussion of 
students’ perceptions. 

2. Evaluation research design. This study insinuates the necessity to make case studies 
close to the professional place in function of some context factors and characteristics of 
students. 

3. Analysis of data. Although naturalistic research will be helpful for developing a 
grounded University teaching subjective theory, a great deal more attention should be 
focused on innovations that do develop professors to high levels of analysis and 
interpretation of educational practice as specific texts and documents. 

4. Development training programs for knowledge and change of professional beliefs. 
Any University should reach a symbiosis between evaluation of teaching, institutional 



ERIC 



13 



evaluation of fields of study and professional development by means of favouring 
development programs based on personal reflection on class teaching. 

5. Formcitive evalucition of teaching innovation. If the hypotheses are true, then we need 
to reconsider what must change if we are to make class innovations more effective 
learning environments. Besides, exercises in teaching innovation should be incorporated 
when evaluating University teaching like sediments where substance and incidental 
teaching and other learning values are revealed. 
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Appendix I 

Evaluation of University Teaching Activities Questionnaire (E. U.T.A.Q.) 

For each sentence select the value of the answer that best suits your perception, draw a 
circle around the your choice on the attached answer sheet. 



Totally agree 


Agree 


I don't know 


Disagree 


Totally 

disagree 


1 


2 


3 ^ 


4 


5 



Dimension A. CLARIFICATION (degree to which University students are given 
explanations, examples and multiple forms of understanding a problem or difficult 
material). 

1. Professor clarifies difficult aspects of this innovative activity. 



2. Professor elaborates the most confusing information of this innovative activity by 
means of outlines, diagrams or illustrations of the main ideas. 

Dimension B. STUDENT AUTONOMY (student perception that University teaching is 
student-centred and that she has been offered the opportunity to make decisions on her 
learning). 

3. This innovative activity has changed my vision on the University student's role. 

4. This innovative activity has changed my attitude towards the subject and the way of 
dealing with University studies. 

5. 1 assume responsibilities in this innovative activity. 

6. 1 suggest possible educational problems and tasks with peers. 

Dimension C. PROFESSOR SCAFFOLDING (degree to which professors demonstrate 
the steps or structure of a problem and provide keys and help to complete the innovative 
activity with success). 

7. This innovative activity gives me keys to solve problems but it doesn't direct me to a 
specific answer. 

8. This innovative activity offers me enough information to be successful. 

9. The professor gives me feedback while I solve a problem in this innovative activity. 
Dimension D. STUDENT PRIOR KNOWLEDGE (degree to which learning activities 
are personally excellent and related to University students’ prior knowledge and 
practical skills). 

10. This innovative activity relates new information to what I have learned previously. 

1 1. 1 use ideas and information that I know to understand something new. 

12. 1 have developed other cognitive capacities in this innovative activity (e.g. analysis, 
synthesis, critical thinking). 

Dimension E. CONNECTIONS (degree to which University students establish their 
own knowledge connections and generate their own learning products). 

13. This innovative activity helps me to investigate, build and relate ideas and facts. 

14. 1 explore how information relates with other topics and subjects. 

Dimension F. INTERROGATION / DISCUSSION (degree to which conjecture, 
questioning, and discussion in this innovative activity is fostered). 

15. This innovative activity encourages University students to ask questions and discuss 
answers given in a book. 

16.1 discuss correct and incorrect solutions to problems. 

Dimension G. EXPLORATIONS BASED IN NEW TECHNOLOGIES (degree to 
which new technological tools and other academic resources facilitate University 
students’ idea generation and knowledge construction). 

17. This innovative activity develops University students’ other study capabilities (e.g. 
handling of tools, document search, library use). 

18. 1 find new information about the topics and subjects using new technologies. 
Dimension H. COLLABORATION AND NEGOTIATION (degree to which University 
students make social interactions with other students to give meanings and obtain 
agreements about teaching activities and viewpoints). 

19. I share ideas, answers and visions with my professor and peers in this innovative 
activity. 

20. 1 learn how to think about a problem from peers and to consider their points of view. 
Dimension I. MOTIVATION (degree to which University students are involved in an 
innovative activity). 

21. 1 am motivated to work in this innovative activity. 

22. This innovative activity improves my opinion about the content of the subject 
(practical vision). 
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23. 1 get more involved in this innovative activity than if I studied it in a theoretical way 
(useful vision). 

Dimension J. EVALUATION (degree to which University students evaluate an 
innovative activity). 

24. 1 believe that this innovative activity develops professors’ interest in teaching. 

25. I believe that innovative activities like this would significantly improve the quality 
of University teaching. 



ANSWER SHEET 



A: CLARIFICATION 








F: INTERROGATION / DISCUSSION 


1.- 1 2 


3 


4 


5 


15.- 1 2 


3 


4 


5 


2.- 1 2 


3 


4 


5 


16.- 1 2 


3 


4 


5 


B: STUDENT AUTONOMY 




G: EXPLORATIONS BASED IN NEW 










TECHNOLOGIES 








3.- 1 2 


3 


4 


5 










4.- 1 2 


3 


4 


5 


17.- 1 2 


3 


4 


5 


5.- 1 2 


3 


4 


5 


18.- 1 2 


3 


4 


5 


6.- 1 2 


3 


4 


5 


















H: COLLABORATION AND 




C: PROFESSOR SCAFFOLDING 


NEGOTIATION 








7.- 1 2 


3 


4 


5 


19.- 1 2 


3 


4 


5 


8.- 1 2 


3 


4 


5 


20.- 1 2 


3 


4 


5 


9.- 1 2 


3 


4 


5 


















I; MOTIVATION 








D: STUDENT PRIOR 


KNOWLEDGE 


















21.- 1 2 


3 


4 


5 


10.- 1 2 


3 


4 


5 


22.- 1 2 


3 


4 


5 


11.- 1 2 


3 


4 


5 


23.- 1 2 


3 


4 


5 


12.- 1 2 


3 


4 


5 


















J: EVALUATION 








E: CONNECTIONS 
























24.- 1 2 


3 


4 


5 


13.- 1 2 


3 


4 


5 


25.- 1 2 


3 


4 


5 


14.- 1 2 


3 


4 


5 












Appendix 2 

Students ’ interview script 

1 . What does teaching mean for you? 

2. What do you think about the teaching style and materials you use in this subject? 

3. What do you believe about your learning? Do you learn more, better, when you work 
individually or in a group? 

4. Do you work in groups in this subject? What does it give to you, and what demands 
does it place on your personal style? 
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5. Point out positive aspects of this subject’s methodology. 

6. Point out negative aspects of this subject’s methodology. 

7. What would you change or how would you believe that you might improve your 
learning? 

8. Score from 1 to 10 the relevance of this subject for your career or future job. Explain 
why? 

9. What do you believe is the reason for the ration between the number of students who 
attend class in this subject and the number registered for it? 

10. Does this subject meet your expectations of it? 

1 1 . In general, are you satisfied with teaching at University? How can you improve it? 

Appendix 3 

Professors ’ interview script 

1 . How can you introduce an innovation of University teaching in the organizational 
culture of your department and College? Please, point out what decisions you would 
make. 

2. Please, indicate how your innovation has affected students: changes in the way they 
study, modification of expectations, solidarity and care in learning, etc. 

3. Did collaboration / cooperation exist with other colleagues of your area, department 
or College before? Please, specify the terms or aspects of the collegiality / cooperation. 

4. What difficulties / rejections, blockades or reticence towards implementing 
innovation have you perceived in the class atmosphere? 

5. What kind of anxieties or satisfactions have you awakened in students and in yourself 
by implementing innovation? 

6. Would you seek to continue with this innovation project next year? Please, give 
reasons. 

7. Have there been changes between the design and report of the innovation project? If 
so, please point out the aspects that have changed and the justifications. 

8. Were you rewarded somehow for the energy and effort made in carrying out the 
innovation? Please, indicate how the rewards provided incentive, or acted as intrinsic / 
extrinsic motivation. 

9. Summarize how many students have been the direct beneficiaries of your innovation. 

10. Conclude making a self-evaluation of your activity in a scale from 1 to 10 (being 1 
totally unsatisfied - I won't repeat this initiative - and 10 - totally satisfied - I’ll repeat 
the innovation and 5 I don't know what I’ll do). 

Appendix 4 



Category System 



CATEGORY 


CODE 


DEFINITION 


PHYSICAL 


EFI 


Physical space where the formative activity is 


ENVIRONMENT 




developed with all its elements (size, distribution, 
light, temperature...). 




ACL 


Social climate - participation, trust or mistrust, etc. - 


CLASS 

ATMOSPHERE 




where a class session is developed. 
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OBJECTIVES / AIMS 
/ EXPECTATIONS 


OME 


What professor and students expect from the 
teaching and learning process. 


PRESENTATION 


EXP 


Educational strategy in which the professor acts as 
a transmitter and the students like receivers of 
information. 


INQUIRY 


IND 


Teaching method based on questions that the 
professor outlines to the students. 


ACTIVITIES 


ACT 


Task collection that a student carries out 
individually. 


GROUP WORK 


TGR 


Task collection carried out by the participation of a 
group of students under a common objective. 


EVALUATION 


EVA 


Measure of attained results. It includes the 
evaluation of the activity, the assessment of 
students, and the evaluation and improvement of 
quality. 


RELATIONSHIP 


REL 


Cordial and friendly relationships between 
professor and students. 


AUDIOVISUAL 

MEDIA 


MAV 


Audio and / or videotape equipment used to 
transmit information to students. 


RESOURCES 


REC 


Written curriculum and communication materials, 
as books, reports, articles, programs... 


COLLABORATION / 
COOPERATION 


CCO 


Interrelation style among students to carry out the 
innovation (among different disciplines, majors, 
courses, groups, people, etc.). 


MOTIVATION OR 
INTEREST 


MIN 


The degree of enthusiasm and expectations that a 
professor and students show in the implementation 
of the innovation. 


VALUES 


VAL 


Development of values by the professor or students 
(solidarity, mutual respect, self-confidence, 
competition, individualism, social concern, 
hierarchy, etc.). 


PROJECTION OR 
IMPLICATIONS 


PIM 


Implications of results or conclusions of the 
innovation to improve teaching, future professional 
life of students, or society. 


TIME ADAPTATION 


ADT 

. 


Appropriate time development of the innovation; 
time invested by a professor and students in the 
innovation. 
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Table I 

Student Demographic Questionnaire (S.D.Q.) (N=665) 



er|c 



Course 

level 



University 

School 



Department 
s and 
subject 
innovations 



First 

61.7% 



Sch. of 
Geograp 
hy and 
History 
(39.2%). 



1 



Gender 



Age 



Type of 
Pre- 

University 

Course 



Type of 
High 
School 
Centre 
where you 
carried out 
Secondary 
Education 



Residence 
during the 
week 



tary 

activities 
while 
studying. 
(125 out of 
665 

answers = 
18,8%) 



Second 

13.4% 



Sch. 0 

Medici 

ne 

(3.6%). 



Sch. of 
Manag 
ement 
(12.6% 
)• 



Third 

17.1% 



Sch. of 

Educatio 

n 

(17.9%). 



Fourth 

3.3% 



Sch. 

of 

Law 

(1.4 

%). 



7 8 



Sch. of 
Psycho 
logy 
(9.2%). 



Male 

31.7% 



Fifth 

.3% 



Sch. of 
Archite 
cture 
(9.2%). 



10 11 



Sch. of 
Techni 
cal 

Archite 

cture 

(9.2%). 
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Female 

68.3% 



18 to 21 year-old 
68.4% 



Sciences 

32.9% 



22 to 23 year- 
old 
18.8% 



Arts 

27.7% 



Public 

74.4% 



Seville-city 

73.2% 



24 to 25 year- 
old 
6.2% 



Mixed Sciences 
6.9% 



Private school 

12.2% 



Village 

25.1% 



older than 25 
years 
6 % 



Mixed Arts 

21.8% 



Subsidised Private 
school 8.4% 



Other towns 



Academic 

background 


3'^'' class 
42.6% 


2"‘' division 
44.1% 


2"'* class 1 
division 
4.1% 


St 


U* class 
2.9% 


Complemen 


You are also studying 


You are preparing for 


You attend some 



another University degree 
2.3% 



State competitive 
examinations) 1.4% 



other course 
15.2% 
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Complemen 
tary jobs 
while 
studying 


You teach children: 

Ihour: .6%; 2hours: 2.3%; 
3hours: 2.1%. 
(39outof665 = 5.9%) 


You work in an 
office: Ihour: .2%; 
2hours: .5%; 3hours: 
.3%. 

(26 out of 665 = 
3.9%) 


You help out at home: 
3.9%; 2hours: 9%; 

3 hours: 8.6%. 

(232 out of 665 = 
34.9%) 


Future 

expectation 

s 


You will work in the State 
administration 
40.9%. 


You wi 
Stai 


1 work outside of the 
te administration 
43.3%. 


The field 
studies at 
this 

University 

were 

chosen as: 


First option 
74.9%. 


Seconc 

19.: 


option 

2%. 


Other options 
4.8%. 


You receive 
some type 
of 

scholarship 
in your 
studies 


Yes 

41.7% 


No 

57.1% 


You repeat 
this subject 


Never 

93.1% 


Once 

5.9% 


More than once 
5% 


You repeat 
subjects of 
other 
courses 


No 

74.6% 


One or two 
14.1% 


More than two 
8.9% 


Housing 
during the 
course 


Home 

65.3% 


Students' residence 
9.5% 


Rented flat 
21.1%. 



* Departments and subject innovations (case frequency and percentage): 

1. History of Art. “Initiation research activity and University teaching”, {n = 254) 
(38.2%). 

2. Modern History. "New teaching strategies in the History of Sciences and 
Technologies” . {n = 7) (1.1%). 

3. Morphological Sciences. "Anatomy of the foot: technical study based on the 
formative and education research", (n = 24) (3.6%). 

4. Company Administrations and Marketing. "Participation approach to enterprise 
administration by means of projections and case studies", (n = 46) (6.9%). 

5. Company Administrations and Marketing. "Development of managerial formation 
tools: the case study method”, {n = 38) (5.7%). 

6. Teaching of Experimental and Social Sciences. "Design of curricular materials for 
teaching and learning Art in Primary Education", (n = 25) (3.8%). 

7. Teaching and School Organization. "Internet applications to preservice teacher 
education”. (N= 43) (6.5%). 

8. Teaching of Mathematics. “New Technologies in the Teaching of Mathematics", {n = 
51) (7.7%). 

9. Roman Law”. "Seminary of exegesis of information sources. Theme : the patrimonial 
situation of family children in Roman Law", {n = 9) (1.4%). 
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10. Psychiatry, Personality, Evaluation and Psychological Treatment. "Role-playing of 
conflicting situations among handicapped students, their parents and the school", (n = 
61) (9.2%). 

11. Architecture Graphic Expression. "The organization of a thematic classroom of 
Architecture as a strategy of education innovation (« = 74) (1 1 . 1 %). 

12. Architecture constructions II. ‘‘Tasks attribution based upon students' learning 
styles of learning. Individualization", (n = 33) (5%). 
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Table II 

Agreement Percentages, Means and Standard Deviations Results for E.U.T.A.Q. 

Dimensions 



Dimensions 


Agreement 

Percentage 


Mean 


S.D. 


Clarification 


35.3 


2.1 


.7654 


Student autonomy 


13.5 


2.8 


.7522 


Professor scaffolding 


20 


2.6 


.6743 


Student prior knowledge 


22.7 


2.1 


.7609 


Connections 


37.6 


2.1 


.7789 


Interrogation / discussion 


23.8 


2.6 


.9859 


Explorations based in new 


26.9 


2.4 


.9378 


technologies 

Collaboration and negotiation 


29.6 


2.3 


1.0321 


Motivation 


21.8 


2.1 


.8319 


Evaluation 


23 


1.8 


.8393 
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Table III 



Analysis of Variance in E.U.T.A.Q. Dimensions for Student Variables in University 
Class Innovations 



Hypothesis 


Dimensions 


F- p 

ratio 


1 . Course level 


Clarification 


16.373 .000 




Student autonomy 


11.839 .000 




Professor scaffolding 


7.687 .000 




Student prior knowledge 


6.15 .000 




Connections 


6.870 .000 




Interrogation / discussion 


9.371 .000 




Explorations based in new 
technologies 


6.480 .000 




Collaboration and negotiation 


24.993 .000 




Motivation 


8.400 .000 




Evaluation 


5.494 .000 


2. University School 


Clarification 


6.691 .000 




Student autonomy 


7.581 .000 




Professor scaffolding 


3.399 .000 




Student prior knowledge 


4.626 .000 




Connections 


5.192 .000 




Interrogation / discussion 


3.186 .000 




Explorations based in new 
technologies 


6.595 .000 




Collaboration and negotiation 


20.914 .000 




Motivation 


4.255 .000 




Evaluation 


5.928 .000 


3. Gender 


Motivation 


2.941 .032 


4. Age 


Professor scaffolding 


3.338 .019 


Interrogation / discussion 


7.702 .000 




Collaboration and negotiation 


10.115 .000 




Motivation 


3.066 .027 


5. Type of Pre-University Course 


Clarification 


5.252 .000 


Professor scaffolding 


2.666 .031 




Connections 


3.014 .017 




Interrogation / discussion 


9.621 .000 




Collaboration and negotiation 


10.028 .000 


6. Academic background 


Clarification 


4.155 .002 




Professor scaffolding 


5.591 .000 




Connections 


3.478 .008 




Interrogation / discussion 


5.136 .000 




Explorations based in new 
technologies 


3.154 .014 


7. Complementary activities while 


Student autonomy 


7.303 .001 


studying 


Professor scaffolding 


5.591 .000 


Student prior knowledge 


4.978 .008 




Connections 


4.749 .010 






BEST COPY AVAILABLE 



II 





Evaluation 


3.886 .022 


8. Complementary jobs while 
studying: You teach children 


Student prior knowledge 


2.766 .019 


9. Complementary jobs while 
studying: You work in an office 


Collaboration and negotiation 


2.620 .022 


10. Complementary jobs while 
studying: You help doing tasks at 
home 


Clarification 
Student prior knowledge 
Collaboration and negotiation 
Motivation 


1.927 .020 
2.691 .001 

1.943 .019 
2.095 .010 


1 1 . Future expectations 


Clarification 


3.735 .024 


12. The field studies at this University 
were chosen . . . 


Evaluation 


3.098 .046 


13. You repeat this subject 


Student autonomy 
Motivation 


6.509 .002 
3.278 .038 


14. You repeat subjects of other 
courses 


Clarification 
Professor scaffolding 
Interrogation / discussion 
Motivation 


10.895 .000 
4.137 .016 
11.971 .000 
4.744 .009 




5 



4.5 
4 

3.5 



3 2 ^- 




Figure L Means on E. U. T.A. Q. Dimensions for all University Class Innovations. 
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PROFESSOR 



Instructional Strategies 



IND, TGR 



Teaching Skills and Techniques 




Planning 

OME 




I 




Classroom teaching 
EXP, ACT, VAL, 
MAV, REC 






r 




Evaluation 

EVA 







College and classroom 
factors 






Context factors 
EFI, ADT 



Climate 
ACL, REL 



Students and 
classroom factors 

CCO, MIN 






Learning results 
PIM 



Figure 2. Conceptual map of class innovation teaching. 
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